Approximating the Permanent with Fractional Belief Propagation

We discuss schemes for exact and approximate computations of permanents, and compare them with each other. Specifically, we analyze the Belief Propagation (BP) approach and its Fractional Belief Propagation (FBP) generalization for computing the permanent of a non-negative matrix. Known bounds and conjectures are verified in experiments, and some new theoretical relations, bounds and conjectures are proposed. The Fractional Free Energy (FFE) functional is parameterized by a scalar parameter $\gamma\in[-1;1]$, where $\gamma=-1$ corresponds to the BP limit and $\gamma=1$ corresponds to the exclusion principle (but ignoring perfect matching constraints) Mean-Field (MF) limit. FFE shows monotonicity and continuity with respect to $\gamma$. For every non-negative matrix, we define its special value $\gamma_*\in[-1;0]$ to be the $\gamma$ for which the minimum of the $\gamma$-parameterized FFE functional is equal to the permanent of the matrix, where the lower and upper bounds of the $\gamma$-interval corresponds to respective bounds for the permanent. Our experimental analysis suggests that the distribution of $\gamma_*$ varies for different ensembles but $\gamma_*$ always lies within the $[-1;-1/2]$ interval. Moreover, for all ensembles considered the behavior of $\gamma_*$ is highly distinctive, offering an emprirical practical guidance for estimating permanents of non-negative matrices via the FFE approach.Comment: 42 pages, 14 figure

Palette-colouring: a belief-propagation approach

We consider a variation of the prototype combinatorial-optimisation problem known as graph-colouring. Our optimisation goal is to colour the vertices of a graph with a fixed number of colours, in a way to maximise the number of different colours present in the set of nearest neighbours of each given vertex. This problem, which we pictorially call "palette-colouring", has been recently addressed as a basic example of problem arising in the context of distributed data storage. Even though it has not been proved to be NP complete, random search algorithms find the problem hard to solve. Heuristics based on a naive belief propagation algorithm are observed to work quite well in certain conditions. In this paper, we build upon the mentioned result, working out the correct belief propagation algorithm, which needs to take into account the many-body nature of the constraints present in this problem. This method improves the naive belief propagation approach, at the cost of increased computational effort. We also investigate the emergence of a satisfiable to unsatisfiable "phase transition" as a function of the vertex mean degree, for different ensembles of sparse random graphs in the large size ("thermodynamic") limit.Comment: 22 pages, 7 figure

Markov entropy decomposition: a variational dual for quantum belief propagation

We present a lower bound for the free energy of a quantum many-body system at finite temperature. This lower bound is expressed as a convex optimization problem with linear constraints, and is derived using strong subadditivity of von Neumann entropy and a relaxation of the consistency condition of local density operators. The dual to this minimization problem leads to a set of quantum belief propagation equations, thus providing a firm theoretical foundation to that approach. The minimization problem is numerically tractable, and we find good agreement with quantum Monte Carlo for the spin-half Heisenberg anti-ferromagnet in two dimensions. This lower bound complements other variational upper bounds. We discuss applications to Hamiltonian complexity theory and give a generalization of the structure theorem of Hayden, Jozsa, Petz and Winter to trees in an appendix

The Fully Frustrated Hypercubic Model is Glassy and Aging at Large DD

We discuss the behavior of the fully frustrated hypercubic cell in the infinite dimensional mean-field limit. In the Ising case the system undergoes a glass transition, well described by the random orthogonal model. Under the glass temperature aging effects show clearly. In the $XY$ case there is no sign of a phase transition, and the system is always a paramagnet.Comment: Figures added in uufiles format, and epsf include

Belief-propagation algorithm and the Ising model on networks with arbitrary distributions of motifs

We generalize the belief-propagation algorithm to sparse random networks with arbitrary distributions of motifs (triangles, loops, etc.). Each vertex in these networks belongs to a given set of motifs (generalization of the configuration model). These networks can be treated as sparse uncorrelated hypergraphs in which hyperedges represent motifs. Here a hypergraph is a generalization of a graph, where a hyperedge can connect any number of vertices. These uncorrelated hypergraphs are tree-like (hypertrees), which crucially simplify the problem and allow us to apply the belief-propagation algorithm to these loopy networks with arbitrary motifs. As natural examples, we consider motifs in the form of finite loops and cliques. We apply the belief-propagation algorithm to the ferromagnetic Ising model on the resulting random networks. We obtain an exact solution of this model on networks with finite loops or cliques as motifs. We find an exact critical temperature of the ferromagnetic phase transition and demonstrate that with increasing the clustering coefficient and the loop size, the critical temperature increases compared to ordinary tree-like complex networks. Our solution also gives the birth point of the giant connected component in these loopy networks.Comment: 9 pages, 4 figure

Survey propagation at finite temperature: application to a Sourlas code as a toy model

In this paper we investigate a finite temperature generalization of survey propagation, by applying it to the problem of finite temperature decoding of a biased finite connectivity Sourlas code for temperatures lower than the Nishimori temperature. We observe that the result is a shift of the location of the dynamical critical channel noise to larger values than the corresponding dynamical transition for belief propagation, as suggested recently by Migliorini and Saad for LDPC codes. We show how the finite temperature 1-RSB SP gives accurate results in the regime where competing approaches fail to converge or fail to recover the retrieval state

Mean-Field Equations for Spin Models with Orthogonal Interaction Matrices

We study the metastable states in Ising spin models with orthogonal interaction matrices. We focus on three realizations of this model, the random case and two non-random cases, i.e.\ the fully-frustrated model on an infinite dimensional hypercube and the so-called sine-model. We use the mean-field (or {\sc tap}) equations which we derive by resuming the high-temperature expansion of the Gibbs free energy. In some special non-random cases, we can find the absolute minimum of the free energy. For the random case we compute the average number of solutions to the {\sc tap} equations. We find that the configurational entropy (or complexity) is extensive in the range T_{\mbox{\tiny RSB}}. Finally we present an apparently unrelated replica calculation which reproduces the analytical expression for the total number of {\sc tap} solutions.Comment: 22+3 pages, section 5 slightly modified, 1 Ref added, LaTeX and uuencoded figures now independent of each other (easier to print). Postscript available http://chimera.roma1.infn.it/index_papers_complex.htm

Locked constraint satisfaction problems

We introduce and study the random "locked" constraint satisfaction problems. When increasing the density of constraints, they display a broad "clustered" phase in which the space of solutions is divided into many isolated points. While the phase diagram can be found easily, these problems, in their clustered phase, are extremely hard from the algorithmic point of view: the best known algorithms all fail to find solutions. We thus propose new benchmarks of really hard optimization problems and provide insight into the origin of their typical hardness.Comment: 4 pages, 2 figure

Computational core and fixed-point organisation in Boolean networks

In this paper, we analyse large random Boolean networks in terms of a constraint satisfaction problem. We first develop an algorithmic scheme which allows to prune simple logical cascades and under-determined variables, returning thereby the computational core of the network. Second we apply the cavity method to analyse number and organisation of fixed points. We find in particular a phase transition between an easy and a complex regulatory phase, the latter one being characterised by the existence of an exponential number of macroscopically separated fixed-point clusters. The different techniques developed are reinterpreted as algorithms for the analysis of single Boolean networks, and they are applied to analysis and in silico experiments on the gene-regulatory networks of baker's yeast (saccaromices cerevisiae) and the segment-polarity genes of the fruit-fly drosophila melanogaster.Comment: 29 pages, 18 figures, version accepted for publication in JSTA

Phase Transitions and Computational Difficulty in Random Constraint Satisfaction Problems

We review the understanding of the random constraint satisfaction problems, focusing on the q-coloring of large random graphs, that has been achieved using the cavity method of the physicists. We also discuss the properties of the phase diagram in temperature, the connections with the glass transition phenomenology in physics, and the related algorithmic issues.Comment: 10 pages, Proceedings of the International Workshop on Statistical-Mechanical Informatics 2007, Kyoto (Japan) September 16-19, 200